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Abstract 

Educator preparation Programs nationwide seek evidence to meet accreditation 
standards for the impact of program completers on P-12 student learning and teaching 
effectiveness. The University of Evansville’s School of Education conducted a 
systematic case study to identify any areas for improvement within the teacher 
preparation program. The effectiveness of program completers’ impact on student 
learning was investigated through student growth data collection, teacher observations, 
and an analysis of results. The purpose of this study was to provide evidence for 
accreditation as well as identify aspects of the teacher preparation program curriculum 


that may need revision or a stronger emphasis. 


Introduction 

The School of Education at the University of Evansville (UE) conducted a case 
study of the Educator Preparation Program (EPP) in an effort to provide evidence for 
CAEP Standards 4.1, Impact on P-12 Student Learning and Development, and 4.2, 
Indicators of Teaching Effectiveness. The case study included evaluation data froma 
sample of program completers who have been teaching for 1 to 5 years. To date this 
case study examined participant data at the end of the 2016-2017 and 2017-2018 
school years and available Indiana Department of Education Teacher Survey data. The 
analysis of instructional practice and student growth data enables the EPP to evaluate 
program effectiveness and the possible need for program curriculum revisions. 

The EPP utilizes educational theory and research-based practices to prepare 
effective classroom teachers. Early field experiences for education majors beginning in 
the very first year provide over 1000 hours of clinical practice prior to graduation. The 
EPP’s commitment to school-university partnerships provides a professional 
development opportunity for all program completers. 

Literature Review 
Theoretical Foundation of Teacher Preparation 

Teacher preparation at the University of Evansville focuses on practical 
experience and mentoring relationships in a professional development school-university 
partnership in an effort to produce positive P-12 student learning outcomes. The 
theoretical foundation of this case study is based on two prominent areas of educational 


research: experiential and social learning, (Bandura, 1977; Dewey, 1938; Rotter 1954; 


Vygotsky, 1978). The premise of these theories, which relate to individuals learning 
from one another to build competencies and confidence, frames this study. 

Dewey (1938) concluded that “all genuine education comes about through 
experience; this does not mean that all experiences are genuinely or equally educative” 
(p. 25). Dewey’s work stressed the importance of the role of quality experiences in 
professional development. Dewey defined learning experiences as a circular pattern of 
trying, questioning, and further experimentation. The foundation of experiential learning 
is that experience matters and without experience there can be no true conceptual 
understanding. The educational goals of institutions of higher education often align with 
the ideals of experiential learning and employ cooperative models for professional 
preparation. Cooperative education allows for the application of knowledge through 
experience and creates an opportunity for growth through communication, reflection, 
and social learning. 

The social learning theory emphasizes the value of observing modeled behaviors 
and attitudes. Rotter’s (1954) work on social learning included the concepts of avoiding 
negative outcomes and promoting positive outcomes through observation of behaviors. 
Modeled behaviors are seen as crucial components to valued and desirable results. 
Early Internship Experiences 

The focus on clinical experiences early and often in teacher preparation at UE 
holds a promise for better prepared novice teachers and a marked improvement for 
student learning outcomes (National Research Council, 2010). The report prepared by 
the National Research Council suggests early clinical preparation is one of the most 


promising practices for effecting student achievement. The early clinical internship 


should be a building block and eventually prepare candidates for a capstone experience 
such as student teaching and a professional portfolio. This portfolio should focus on the 
candidate’s development, but also include the ultimate goal of impact on student 
learning. These experiences are even more beneficial to teacher candidates when they 
include supervision and frequent feedback. 

Historically, the role of a mentor teacher throughout the internship experience 
was to accept the teacher candidate into the classroom, serve as host, and model 
teaching practices. A significant shift in the roles and responsibilities of mentor teachers 
occurred over the past three decades (Ambrosetti & Dekkers, 2010). The mentor 
teacher has long been credited with playing the most influential and significant role in 
the teacher candidates’ professional development (Izadinia, 2016; Karmos & Jacko, 
1977; Klieger & Oster-Levinz, 2015; Roland & Beckford, 2010). In an early pivotal study 
by Karmos and Jacko (1977) on the perspectives of teacher candidates on teacher 
preparation, mentor teachers were considered the most significant and positive 
influence on teacher candidate learning by more than half of the respondents. The 
influence was credited to the role mentor teachers played in the development of teacher 
candidates’ knowledge, skills, and professional attitudes. The research on the 
importance of early practical experience and the influence of the mentor teacher paved 
the way for new designs in teacher preparation programming. 

Professional Development School Model 

The professional development school (PDS) model, which began in the 1990s, is 

currently being practiced within teacher education. This transformation posits that 


effective teacher preparation cannot solely rest on the shoulders of teacher preparation 


programs alone. The professional development school consists of an innovative design 
formed through partnerships between teacher preparation programs and P—12 schools. 
This model has several goals: collaboration and a symbiotic partnership between 
schools and universities, expanded early clinical internship experiences, reform in 
teacher education, enhanced student achievement, professional development for 
participants, and research on promoting cultural and linguistic diversity and culturally 
responsive teaching in preparing teachers for urban school settings (Darling-Hammond, 
2009; Grossman, Hammerness, & McDonald, 2009; Johnston-Parsons, 2012). 
Research on professional development partnerships indicates lower attrition among 
new teachers and greater teaching efficacy (Darling-Hammond, 2005). 

The concept of professional development schools reorganized the teacher 
candidate field experiences. Criticism for the divide between the educational theory 
taught at the university and the integration of these theories into classroom teaching 
was addressed by the expectations of the professional development model. The 
collaborative nature of the professional development model sought to provide 
consistency and congruence between the university program and classroom clinical 
expectations (Zeichner, 2010). The goal was to create clinical settings that would 
enable teacher candidates to acquire essential skills. The model was designed to link 
theory and practice by bringing university instruction into the schools and allowing for 
immediate application in the classroom setting. The role of the mentor teacher became 
an integral part of this ideology. The findings of a study by Klieger and Oster-Levinz 
(2015) revealed teachers who are now serving as mentors credit their participation ina 


professional development teacher preparation model for a greater understanding of the 


mentor role. Mentor teachers were now invited to share in the experience of planning 
and problem solving. The passive manner of including mentor teachers in teacher 
preparation had taken the shift to an increasingly active and engaging partnership. One 
of the crucial roles of the mentor teacher under this new model of teacher preparation 
was to assist the teacher candidate in the ability to link the art of reflection with 
professional practice in an effort to improve student outcomes. 

Johnston-Parsons (2012) offered suggestions for implementing a successful 
teacher preparation model through partnerships, like the professional development 
school model, in teacher preparation. Johnston-Parsons identified that an essential key 
to success is the mutual ownership of the learning community. Additionally, 
collaborative roles need to be established and well defined and the relationship should 
be built on trust and offer benefits to all stakeholders. The community of practice 
created by the professional development school should be theoretically grounded in 
social learning theory (Bandura, 1977). The theory suggests we are social beings and 
that Knowledge and learning are gained through observing and experiencing the world 
around us. During teacher preparation, this occurs when teacher candidates have the 
Opportunity to practice teaching skills and observe mentors in the field. 

In summary, this case study was designed to reflect the theories of experiential 
learning and social learning used within teacher preparation at the University of 
Evansville. The decades of research have provided ample evidence of the contributions 
of these educational theories in the professional preparation of effective teachers and 
ultimately P-12 student achievement. Relevant research in early and rigorous Clinical 


experiences, the significant role of the mentor teacher, and the importance of a 


professional development school model frame the design of teacher preparation at the 
University of Evansville. 
Methodology 

The researchers used a quasi-experimental quantitative design to conduct the 
multi-year case study. This was a logical approach based on the numerical data 
provided by case study participants and IDOE surveys (Mertens, 2010). The data 
collected provided direct evidence of impact on student learning and teacher 
effectiveness. 
Setting and Participants 

Participants for this case study were solicited from a pool of program completers 
employed in the largest regional employer of the School of Education program 
completers from the University of Evansville. According to the Associate Superintendent 
for Strategy and Accountability, teachers employed in the school corporation may share 
their evaluation data with their EPP. Faculty from the School of Education held a focus 
group with five completers to gain more knowledge on the iObservation instrument and 
the locally developed assessment (LDA) used across the school corporation. They 
discussed the items on each instrument and how this information was used. As a result, 
the University of Evansville School of Education program completers, working in 
classrooms for five years or less, were asked to share their evaluation data with the 
School of Education. It was determined that this service period in the classroom would 


be an appropriate measure of EPP influence on teaching practices. 


Data Collection 

The school Corporation’s online Personnel Directory was used to identify the 
program completers and their respective school sites. Invitations to participate in the 
case study were mailed to all program completers within the last 5 years, who were 
currently employed in the district. Those interested in participating in the case study 
signed and returned a permission to use data agreement. This form granted the EPP 
permission to use instructional practice and student growth data solely for accreditation 
purposes and ensured confidentiality. The evaluations could be mailed, if the 
participants wished to remain anonymous. It was requested that all students’ names be 
removed. After meetings with their Associate Superintendent for Strategy and 
Accountability, it was determined that the School of Education should obtain copies of 
our graduates’ iObservations and locally developed assessment data to determine their 
impact on P-12 student achievement. Therefore, participants were asked to share both 
teaching observation evaluations and student growth data. Eleven program completers 
returned signed agreements. To date eleven participants shared evaluation documents 
for the 2016-2017 school year, and six participants shared evaluation documents for the 
2017-2018 school year. Two participants shared date from multiple school years, and 
some did not yet have all student growth data available. All of the participants shared 


both the iObservation evaluations and grade level appropriate student growth data. 


Measures 

iObservations. 

According to information obtained from the Teachers Association and verified by 
School of Education program completers during a focus group, teaching observations 
and development data account for 90% of the teachers’ summative rating. Observations 
are recorded in iObservation, an online software platform that serves as an interface for 
evaluation data for both teachers and administrators. Every teacher has a login and 
password that provides access to evaluation data and an opportunity to upload 
evidence or artifacts, to support their instructional practice. Each teacher is observed by 
a minimum of two times over the course of a school year by two different evaluators. 
Observations may be conducted by building-level administrators including principals, 
assistant principals, department chairs, or professional development specialists. They 
may also be observed by district-level administrators, including directors, assistant 
directors, or coordinators. 

Evidence gathered for the 2016-2017 academic year from the observations and 
teacher-provided artifacts were scored according to the evaluation rubric. Teachers are 
assigned a rating on a 5-point scale for each of the 4 domains of essential 
competencies as well as all competencies for which evidence was gathered via 
observation or teacher-provided artifacts. If non-essential competencies are not scored, 
their weight is distributed proportionally among the other competencies within the same 
domain. Based on the percentage weighing for each competency as indicated in the 


outlines, and overall rubric score between 1 and 5 is assigned to each teacher. A score 


of a 3 on the rubric represents solid, effective teaching. The rubric domains of 
competencies are defined and weighted as follows: 
Domain 1: Instructional Design and Assessment (20% of overall score) 
e Lesson design 
e Standards-based instructional goals and learning outcomes 
e Multiple assessments 
Domain 2: Instructional Delivery (55% of overall score) 
e Academic feedback 
e Teacher knowledge of students 
Domain 3: Learning Environment (15% of overall score) 
e Managing student behaviors 


e Environment 


Domain 4: Collaboration and Professional Responsibilities (10% of overall score) 


e Collaboration with colleagues 

e Continuous professional skills and knowledge development 
e Awareness and advocacy for profession and students 

e Stakeholder engagement and communication 

e Teacher leadership 


e Teacher compliance to policy and procedures 


In simplest terms, the teachers are evaluated on planning for instruction, content 
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delivery and assessment, a classroom environment conducive to learning, collaboration, 


and professionalism. 
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Evidence gathered for the 2017-2018 academic year from the observations and 
teacher-provided artifacts were scored according to the revised evaluation rubric. 
Teachers are assigned a rating on a 5-point scale for all competencies. The revised 
rubric domains of competencies are defined and weighted as follows: 

Domain 1: Instructional Design and Assessment (15% of overall score) 
e Effective lesson design 
e Assessment 
Domain 2: Instructional Delivery (30% of overall score) 
e Resources, activities, and materials 
e Presenting instructional content 
e Lesson pacing and structure 
e Questioning strategies 
e Thinking and problem solving 
e Monitoring for learning 
e Knowledge of students as learners 
Domain 3: Learning Environment (10% of overall score) 
e Respectful culture 
e Environment 
Domain 4: Collaboration and Professional Responsibilities (45% of overall score) 
e Collaboration to support learning 


e Professionalism 


12 


Student Growth Data. 

Under the teacher evaluation system, student data will account for 10% of the 
final summative rating. For the majority of teachers, that 10% will include at least two of 
the following factors: ISTEP student growth data and LDA data (4%), performance 
toward school or corporation school plan improvement (SIP) goals (2%), and school 
grade (4%). 

For most teachers LDAs will be the work of their professional development 
community (PLC), and the development of these assessments will occur during the first 
part of the school year. LDAs will be teacher created and peer reviewed. Information on 
LDAs is available on the teacher portal. Effectiveness is rated on a 5-point scale based 
on the percentage of students showing improvement from pre- to post- test. 
Effectiveness data are scored as follows: 

1 - Less than 20% 
2 - 20-29.9% 

3 - 30-59.9% 

4 - 75% or more 

Surveys. 

The IDOE provided data from valid, reliable and structured observation 
instruments that demonstrate completer ability to effectively apply professional 
knowledge, skills and dispositions in classroom settings. Indiana Code 20-28-11.5-9, 
Sec. 9 requires school corporations to share the results of the most recent teacher 
evaluations for those with three or fewer years of teaching experience with EPP’s. 


Principals in each school corporation are required to complete a survey that provides 


information regarding the principal's assessment of the quality of instruction by each 
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teacher preparation program in Indiana for their teachers employed at the school who 
initially received their teaching license in Indiana in the previous two (2) years. 
Data Analysis 

Data were analyzed using SPSS statistical software. Data screening anda 
descriptive statistics analysis were conducted to provide the range, mean, and standard 
deviation for each measure. These analyses were carried out to assess the effect of the 
EPP on programs completers’ teaching effectiveness and impact on P-12 student 
learning. 

Limitations 

The quasi-experimental design in educational research restricted random 
sampling and may have contributed to a threat of self-selection reliability. There was 
also a risk of self-selection bias, as the program completers were invited to participate in 
the study. It is important to note that at the beginning of the 2017-2018 academic year 
the evaluation rubric was revised. The essential competencies were eliminated and all 
competencies were scored. The weights for each of the domains were also revised. 

At this point in the data collection process, the condensed time frame of only two 
academic years and a relatively small sample size also presented possible limitations to 
the case study. The above-mentioned limitations posed threats to the validity and 
reliability of the case study and warrant on-going research. 

Findings 
Program Completer Data 2016-2017 
Data Scale 
5.0 - 3.75 Highly Effective 


3.74 — 2.75 Effective 
2.74 — 1.75 Needs Improvement 
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1.74 — 1.0 Ineffective 


*LDA — Values represent % of students showing improvement form pre-post 
assessment 


Descriptive statistics (N=11) from the iObservation evaluation instrument for 
Instructional Practice revealed a range of 3.00(Effective)-4.66(Highly Effective) with 
mean rating of 3.62(Effective) and .54 standard deviation. Descriptive statistics (N=10) 
from the Student Growth data through ISTEP or SIP revealed a range of 
1.00(Ineffective)-5.00(Highly Effective) with a mean rating of 2.93(Effective) and 1.82 
standard deviation. Student Growth data through LDA(N=10) revealed a student 
improvement range of 68%-100%. These growth percentages are interpreted as a 
range of 4.00-5.00(Highly Effective) on the rating scale with a mean rating of 
4.89(Highly Effective) and 1.27 standard deviation. The participants received a student 
data rating based on both the ISTEP/LDA, SIP data, and school grade. The Student 
Data Rating revealed a range of 2.80(Effective)-4.50(Highly Effective) with a mean 


rating of 3.39(Effective) and .64 standard deviation. 


Program Completer Data 2017-2018 


Descriptive statistics (N=3) from the iObservation evaluation instrument for 
Instructional Practice revealed a range of 4.18(Effective)-4.79(Highly Effective) with 
mean rating of 4.46(Effective) and .30 standard deviation. Descriptive statistics (N=1) 
from the Student Growth data through SIP revealed a range of 1.00(Ineffective)- 
1.00(Ineffective). Student Growth data through LDA(N=6) revealed a student 
improvement range of 87%-100% with a mean growth percentage of 96.50%. These 


growth percentages are interpreted as a range of 4.00-5.00(Highly Effective) on the 


15 


rating scale. The participants received a student data rating based on both the 
ISTEP/LDA, SIP data, and school grade. The Student Data Rating revealed a range of 


3.00(Effective). 


Examining the mean scores for each of the four Domains in the iObservation 
instructional practice evaluation revealed “effective” and “highly effective” ratings. All 
individual participants were rated “effective” and “highly effective” across all four 
Domains with the exception of one participant who was rated “needs improvement” for 
Domain 3, Learning Environment. Overall, the highest mean score was in Domain 3, 
Learning Environment. The second highest mean score was in Domain 2, Collaboration 
and Professional Responsibilities, followed by Domain 1, Instructional Design and 


Assessment, and finally Domain 2, Instructional Practice. 


According to the Indiana Department of Education Teacher Survey Data, the 
percent of University of Evansville program completers receiving a teacher evaluation 


rating of Effective or Highly Effective during the 2014-2015 school year are as follows: 


Teachers with One (1) Year Experience = 21 (18 Effective, 2 Highly Effective) 
Teachers with Two (2) Years of Experience = 21 (16 Effective, 5 Highly Effective) 
Teachers with Three (3) Years of Experience = 25 (10 Effective, 10 Highly Effective) 
Grand Total Rated Effective: 44 

Grand Total Rated Highly Effective: 17 

Grand Total Effective and Highly Effective: 61 


Grand Total Teachers Evaluated: 67 
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The data indicated that 97.6% of program completers with 1-2 years of teaching 
experience were rated either "Effective" or "Highly Effective" as a practicing teacher. 
Ninety-one percent of program completers with 1-3 years of teaching experience were 


rated either “Effective” or “Highly Effective” as a practicing teacher. 


The percentage of the University of Evansville program completers receiving a 
teacher evaluation rating of Effective or Highly Effective during the 2015-2016 school 


year are as follows: 


Teachers with One (1) Year Experience = 28 (26 Effective) 

Teachers with Two (2) Years of Experience = 17 (13 Effective, 2 Highly Effective) 
Teachers with Three (3) Years of Experience = 25 (18 Effective, 7 Highly Effective) 
Grand Total Rated Effective: 57 

Grand Total Rated Highly Effective: 9 

Grand Total Effective and Highly Effective: 66 


Grand Total Teachers Evaluated: 70 


The data indicated that 91.1% of program completers with 1-2 years of teaching 
experience were rated either "Effective" or "Highly Effective" as a practicing teacher. 
Ninety-four percent of program completers with 1-3 years of teaching experience were 


rated either “Effective” or “Highly Effective” as a practicing teacher. 
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Discussion and Recommendations 


This longitudinal case study allowed the EPP to continually monitor the teaching 
effectiveness of program completers in an effort to analyze program curriculum and 
design. The findings from this case study provided evidence of program completers for 
CAEP Standards 4.1, Impact on P-12 Student Learning and Development, and 4.2, 
Indicators of Teaching Effectiveness. The participant and IDOE survey data indicated all 
program completers are rated either “effective” or “highly effective” for student learning 
outcomes and teaching practices. The findings also revealed that the program 
completer ratings improved over time. 

The findings suggest the program completers’ immersion in the K-12 classroom 
early and often during their teacher preparation program contributed to effective 
classroom teaching (Darling-Hammond, 2009). In addition, bridging theory and practice 
was Critical for the process of preparing effective teachers (National Research Council, 
2010; Zeichner, 2010). The collaborative partnership between the EPP and a local 
school corporation was crucial for creating these positive outcomes for both program 
completers and K-12 students (Johnston-Parsons, 2012). 

The practical application of skills during several clinical experiences proved to be 
very beneficial to program completers across all disciplines. The design of the teacher 
preparation program allowed teacher candidates to learn strategies and methods that 
could immediately be applied in a clinical setting. These opportunities to practice in the 
actual classroom setting proved to be an asset for early career practicing teachers. 

Discussion with focus group participants identified the key reasons for 


practitioner success as the wide variety of experiential learning working closely with 
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mentor teachers and a teacher preparation program that allowed teacher candidates to 
build the skill sets required to be effective classroom teachers. The study participants 
have the confidence and skill set to work in culturally diverse inner-city schools. All 
program completers have had experience in Title | schools. Due to the coursework and 
emphasis on variety of school community experiences, program completers are 
culturally competent. The findings confirmed program completers are effectively 
teaching in a highly diverse student population. 

The case study revealed some limitations that may be addressed through 
continued research. Securing additional years of data and soliciting more study 
participants are among the recommendations for future ongoing research and data 
analysis. Continued analysis of program completer data will enable the EPP to identify 
any possible areas of improvement in the teacher preparation program. The four 
domains of the iObservation teacher evaluation instrument, Instructional Design and 
Assessment, Instructional Delivery, Learning Environment, and Collaboration and 
Professional Responsibilities, provided specific areas to examine for program 


improvements and curriculum revision. 
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